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DETAILED ACTION 

1 . This communication is in response to the Amendments and Arguments filed on 
05/14/2008. Claims 1-22 remain pending and have been examined. The Applicants' 
amendment and remarks have been carefully considered, but they do not place the 
claims in condition for allowance. Accordingly, this action has been made FINAL. 

2. All previous objections and rejections directed to the Applicant's disclosure and 
claims not discussed in this Office Action have been withdrawn by the Examiner. 

Change of Examiner 

It should be noted that the Examiner of record for this Application has changes 
from Benjamin Gaddy to Paras Shah. 

Response to Arguments 

3. Applicant's arguments (pages 9-1 3) filed on 05/1 4/2008 with regard to claims 1 - 
22 have been fully considered but they are moot in view of new grounds for rejection. 

Response to Amendment 

4. Applicants' amendments filed on 05/14/2008 have been fully considered. The 
newly amended limitations in claims 1,17, and 21 necessitate new grounds of rejection. 
The newly added limitation in the specified claims includes "wherein the media 
fragments are not audio media fragments and wherein the media fragments are 
unrelated to the audio signal prior to the obtaining act..." 
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Claim Rejections - 35 USC §112 

5. The following is a quotation of the first paragraph of 35 U.S. C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

6. Claim 21 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one 
skilled in the relevant art that the inventor(s), at the time the application was filed, had 
possession of the claimed invention. Specifically, the claim limitation of "computer 
readable medium" has not been described in the Specification as originally filed and 
hence constitutes new matter. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 1-3, 5, 7, 9, 10, 16-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Finn (WO 01/1 1495) in view of Mitton (US 6,355,869). 

As to claims 1, 17, 21, and 22: Finn discloses augmenting an audio signal (see 
Figure 1) comprising: 
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receiving an audio signal (Figure 2, input search criteria steps 20 and 21) 

extracting features from said audio signal (see Figure 2 step 22, identify 
pitch of successive notes), 

generating a time based table of dramatic parameters according to the 
extracted features (see page 6, 3rd and 4th paragraph, and see Figure 3A, from 
the cited portion and the figure it can be seen that a time ordered sequence of 
pitches are extracted), and 

obtaining media fragments at least in part in dependence on the table of 
dramatic parameters (see page 15, line 10-33, comparison between query and 
database based on similarity) files, wherein the media fragments are not audio 
media fragments and wherein the media fragments are unrelated to the audio 
signal prior to the obtaining act (see page 3, lines 5-7, only a search criteria in 
input and a target file is retrieved, which is not the same as the audio input 
above), and 

outputting said media fragments (see page 21, lines 21-28, music file 
output or list displayed to user) 

However, Finn does not specifically teach a time-ordered table. 

Mitton does teach a time- ordered table (see Col. 5, lines 12-22, where 
Mitton discusses a pseudo wave file with a series of pitch coefficients for each 
frame and Figure 33). 

It would have been obvious to one skilled in the art at the time the 
invention was made to modify the audio search as taught by Finn, and use a 
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time-ordered table as taught by Mitton, thus allowing a user to produce a musical 
score from a recording, as discussed by Mitton (see Col. 1, lines 55-60). 

As to claim 2, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

Furthermore, Finn teaches features extracted from said audio signal 
include tempo (see page 25, lines 2, 15, key and tempo determined from the 
input and is used in search criteria (see page 23 lines 22-26, used in first pass 
matching) . 

As to claim 3, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

Furthermore, Mitton does teach generation of a time- ordered table (see 
Col. 5, lines 12-22, where Mitton discusses a pseudo wave file with a series of 
pitch coefficients for each frame and Figure 33). 

Furthermore, Finn discloses the table of dramatic parameters comprises 
retrieving a list of dramatic parameters and associated audio features (see page 
1 1 , lines 8-10, features from data are compared with search criteria, where the 
matching criteria of dramatic parameters is shown in page 25, lines 2 and 15), 
comparing and matching the extracted features with the retrieved associated 
audio features (see page 1 1 , lines 8-10, features from data are compared with 
search criteria), and inserting an entry comprising the dramatic parameter 
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associated with the audio feature (see page 23, lines 21 -page 25, lines 16, 
various criteria are determined in order to determined match, where the 
determination of the dramatic parameter is the inserting for matching purposes.) 

As to claim 5, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

Furthermore, Finn teaches obtaining said media fragments comprises 
selecting a fragment from a store (see page 11, lines 8-1 0,m music files in 
database 9 and 10 used), said fragment being stored with an associated 
dramatic parameter which matches the respective entry in the table of dramatic 
parameters (see page 25, lines 2, 15, key and tempo determined from the input 
and is used in search criteria (see page 23 lines 22-26, used in first pass 
matching) . 

As to claim 7, Finn in view of Mitton teach all of the limitations as in claim 5, 

above. 

Furthermore, Finn teaches receiving user input, said user input affecting 
said obtaining (see page 6, lines 13-18, user inputs a voice or a tune and see 
page 3, lines 1-8, based on user input a matching music is obtained). 

As to claim 9, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 
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Furthermore, Finn teaches Furthermore, Finn teaches further comprising 
storage for storing said media fragments (see page 1 1 , line 9, database 9 or 10). 

Furthermore, it would have been obvious to one of ordinary skilled in the 
art to have stored the audio signal at least temporarily as well in order to perform 
the extraction of features from the audio signal for comparison (see Finn, page 
6, lines 15-19). 

As to claim 1 0, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

Furthermore, Finn teaches wherein said outputting comprises rendering 
said media fragments and said audio signal (see page 21 , lines 26-29, link to the 
media fragment is displayed which the user can select to hear. It is obvious that 
the computer system consists of a built in speaker to hear such results 
corresponding to the tune of the search query. Hence, the rendering of the audio 
signal occurs by the rendering of a match that is found similar to the tune that 
was input.) 

As to claim 1 6, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

Furthermore, Finn teaches wherein combinations of extracted features 
have associated dramatic parameters (see page 25, line 15 and 16, where the 
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tempo is based on mean note durations in seconds, i.e., the mean of the pitches 
for a specific duration determines the dramatic parameter). 



As to claims 18 and 22, Finn in view of Mitton teach all of the limitations as in 
claim 17 and 21, above. 

Furthermore, Finn teaches further comprising storage for storing said 
media fragments (see page 1 1 , line 9, database 9 or 1 0). 

Furthermore, Mitton teaches storing the dramatic parameters ( see col. 5, 
lines 22-32, list of event and lines 35 where the MIDI file is created and is obvious 
it will be stored (see Abstract). 

As to claim 1 9, Finn in view of Mitton teach all of the limitations as in claim 1 7, 

above. 

Furthermore, Finn teaches wherein said at least one output device 
comprises display means on which said media fragments are displayed (see 
page 6, lines 5, monitor 4, and page 21 , lines 25-26, user presented with search 
results.). 



As to claim 20, Finn in view of Mitton teach all of the limitations as in claim 1 7, 
above.. 

Furthermore, Finn teaches wherein said at least one output device 
comprises display means on which said media fragments are displayed (see 
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page 6, lines 5, monitor 4, and page 21, lines 25-26, user presented with search 
results.). 

Furthermore, Mitton teaches the output device responsive to instructions 
associated with said dramatic parameters (see col. 10, lines 6-9, user can modify 
the event list). 

9. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Finn in 
view of Mitton as applied to claim 1 above, and further in view of Weare (US 
2003/0045954). 

As to claim 4, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

However, Finn in view of Mitton does not teach the parameters being 
mood, change of pace incidents. 

Weare does teach use of parameters mood (see [0095], mood), change of 
pace (see [0066], flow)) and incidents (see [0066], rhythmic activity]). 

It would have been obvious to one skilled in the art at the time the 
invention was made to modify the audio searching of Finn in view of Mitton, and 
use video as taught by Weare, for the classification of media entities according to 
melodic properties (see Weare [0002]). 
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10. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over Finn in 
view of Mitton as applied to claim 1 above, and further in view of Kodama (US 
2002/0106127). 

As to claim 8, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

However, Finn in view of Mitton do not specifically teach the media 
fragments being video data. 

Kodama teaches video (see Abstract, movie image data). 

It would have been obvious to one skilled in the art at the time the 
invention was made to modify the audio searching of Finn in view of Mitton, and 
use video as taught by Kodama, for effectively retrieving movie information in a 
multimedia system (see Kodama [0003]). 

11. Claims 6 and 11-14 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Finn in view of Mitton as applied to claim 1 above, and further in view of Balnaves 
(US 6,954,894). 

As to claim 6, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

However, Finn in view of Mitton do not specifically teach generating 
fragments. 
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Balnaves teaches generating a fragment (see col. 1 1 , lines 12-29, where 
the user input is modified to form a fragment depending on template selected, 
silent movie is chosen). 

It would have been obvious to one skilled in the art at the time the 
invention was made to modify the audio searching of Finn in view of Mitton, and 
use video as taught by Balnaves, for effectively controlling and editing multimedia 
output (see Balnaves , col. 1 , lines 7-1 1 ). 

As to claim 1 1 , Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

Furthermore Finn in view of Mitton teach dramatic parameter data, 
matching dramatic parameters to media fragments, and selecting and generating 
according to dramatic parameter lists. 

However, Finn in view of Mitton do not specifically teach the story 
template 

Balnaves teaches prior to obtaining said media segments, selecting a 
story template (see col. 8, lines 27-30, user selects template) at least in part in 
dependence on said table of dramatic parameter (see col. 8, lines 54-60, 
templates used to evoke action or intent and see Figure 12 and 13, where each 
type of movie has a specific template), said story template affecting said 
obtaining of media fragments (see Figure 5, 501 and 508, template and movie 
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player, output of processed data) (e.g. The template chosen affects the output 
data). 

It would have been obvious to one skilled in the art at the time the 
invention was made to modify the audio searching of Finn in view of Mitton, and 
use video as taught by Balnaves, for effectively controlling and editing multimedia 
output (see Balnaves , col. 1, lines 7-11). 



As to claim 12, Finn in view of Mitton in view of Balnaves teach all of the 
limitations as in claim 1, above. 

Furthermore, Finn teaches the sue of dramatic parameters (see page 6, 
3rd and 4th paragraph, and see Figure 3A, from the cited portion and the figure it 
can be seen that a time ordered sequence of pitches are extracted) 

Furthermore, Balnaves teaches wherein said story template comprises 
dramatic parameter data related to a narrative story structure (see Figure 12 and 
Figure 13, each type of template movie selected consists of various parameters. 

As to claim 13, Finn in view of Mitton in view of Balnaves teach all of the 
limitations as in claim 1, above. 

Furthermore, Finn teaches matching the dramatic parameters with the 
media fragments features (see page 1 1 , lines 8-10, features from data are 



Application/Control Number: 10/540,315 Page 13 

Art Unit: 2626 

compared with search criteria, where the matching criteria of dramatic 
parameters is shown in page 25, lines 2 and 15), 

Furthermore, Balnaves teaches using a story template comprises dramatic 
parameter data related to a narrative story structure (see Figure 12 and Figure 
13, each type of template movie selected consists of various parameters. 

As to claim 14, Finn in view of Mitton in view of Balnaves teach all of the 
limitations as in claim 1, above. 

Furthermore, Balnaves teaches wherein the story template for selection is 
generated according to logical story structure rules and the dramatic parameter 
list (see Figures 12 and 13, where sample template is shown) (e.g. From the 
Figure, it is obvious to one skilled in the art that the templates were generated 
beforehand. Based on movie genre or user preferences related to the genre.) 

12. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Finn in 
view of Mitton as applied to claim 1 above, and further in view of Williams (US 
6,308,154). 

As to claim 15, Finn in view of Mitton teach all of the limitations as in claim 1 , 

above. 

However, Finn in view of Mitton do not specifically teach the use of 
physical markup language tags. 
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Williams teaches instruction set of a markup language (see Col. 3, lines 2- 
8, where Williams discusses attributes are encoded using a markup language 
and markup indicators). 

It would have been obvious to one skilled in the art at the time the 
invention was made to modify the audio searching as taught by Finn in view of 
Mitton, and use instruction set of a markup language as taught by Williams, thus 
allowing measurement and encoding of recognized content, as discussed by 
Williams (see Col. 1 , lines 52-57). 

Conclusion 

1 3. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See M PEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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14. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Negishi (US 6,504,089) is cited to disclose music searching based on scoring like 
music. Woo (Us 6,678,680) is cited to disclose a music search engine based on note 
differences. Hinman (US 7,022,905) is cited to disclose a classification and searching of 
musical information. Willmot et al. (US 2002/0147592) is cited to disclose searching of 
speech and retrieving relevant segments. Gayama (US 2004/0144238) and Ozick (US 
2003/0233930) is cited to disclose music searching using chord progression 
information. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to PARAS SHAH whose telephone number is (571)270- 
1650. The examiner can normally be reached on MON.-THURS. 7:00a. m.-4:00p.m. 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571)272-7603. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



IP. S.I 

Examiner, Art Unit 2626 

07/14/2008 

/Patrick N. Edouard/ 

Supervisory Patent Examiner, Art Unit 2626 



